Assessment In Medical Education Research Articles

Associations of Internal Medicine Residency Milestone Ratings and Certification Examination Scores With Patient Outcomes.

Despite its importance to medical education and competency assessment for internal medicine trainees, evidence about the relationship between physicians' milestone residency ratings or the American Board of Internal Medicine's initial certification examination and their hospitalized patients' outcomes is sparse. To examine the association between physicians' milestone ratings and certification examination scores and hospital outcomes for their patients. Retrospective cohort analyses of 6898 hospitalists completing training in 2016 to 2018 and caring for Medicare fee-for-service beneficiaries during hospitalizations in 2017 to 2019 at US hospitals. Primary outcome measures included 7-day mortality and readmission rates. Thirty-day mortality and readmission rates, length of stay, and subspecialist consultation frequency were also assessed. Analyses accounted for hospital fixed effects and adjusted for patient characteristics, physician years of experience, and year. Certification examination score quartile and milestone ratings, including an overall core competency rating measure equaling the mean of the end of residency milestone subcompetency ratings categorized as low, medium, or high, and a knowledge core competency measure categorized similarly. Among 455 120 hospitalizations, median patient age was 79 years (IQR, 73-86 years), 56.5% of patients were female, 1.9% were Asian, 9.8% were Black, 4.6% were Hispanic, and 81.9% were White. The 7-day mortality and readmission rates were 3.5% (95% CI, 3.4%-3.6%) and 5.6% (95% CI, 5.5%-5.6%), respectively, and were 8.8% (95% CI, 8.7%-8.9%) and 16.6% (95% CI, 16.5%-16.7%) for mortality and readmission at 30 days. Mean length of stay and number of specialty consultations were 3.6 days (95% CI, 3.6-3.6 days) and 1.01 (95% CI, 1.00-1.03), respectively. A high vs low overall or knowledge milestone core competency rating was associated with none of the outcome measures assessed. For example, a high vs low overall core competency rating was associated with a nonsignificant 2.7% increase in 7-day mortality rates (95% CI, -5.2% to 10.6%; P = .51). In contrast, top vs bottom examination score quartile was associated with a significant 8.0% reduction in 7-day mortality rates (95% CI, -13.0% to -3.1%; P = .002) and a 9.3% reduction in 7-day readmission rates (95% CI, -13.0% to -5.7%; P < .001). For 30-day mortality, this association was -3.5% (95% CI, -6.7% to -0.4%; P = .03). Top vs bottom examination score quartile was associated with 2.4% more consultations (95% CI, 0.8%-3.9%; P < .003) but was not associated with length of stay or 30-day readmission rates. Among newly trained hospitalists, certification examination score, but not residency milestone ratings, was associated with improved outcomes among hospitalized Medicare beneficiaries.

Physical and biophysical markers of assessment in medical training: A scoping review of the literature

Purpose Assessment in medical education has changed over time to measure the evolving skills required of current medical practice. Physical and biophysical markers of assessment attempt to use technology to gain insight into medical trainees’ knowledge, skills, and attitudes. The authors conducted a scoping review to map the literature on the use of physical and biophysical markers of assessment in medical training. Materials and methods The authors searched seven databases on 1 August 2022, for publications that utilized physical or biophysical markers in the assessment of medical trainees (medical students, residents, fellows, and synonymous terms used in other countries). Physical or biophysical markers included: heart rate and heart rate variability, visual tracking and attention, pupillometry, hand motion analysis, skin conductivity, salivary cortisol, functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS). The authors mapped the relevant literature using Bloom’s taxonomy of knowledge, skills, and attitudes and extracted additional data including study design, study environment, and novice vs. expert differentiation from February to June 2023. Results Of 6,069 unique articles, 443 met inclusion criteria. The majority of studies assessed trainees using heart rate variability (n = 160, 36%) followed by visual attention (n = 143, 32%), hand motion analysis (n = 67, 15%), salivary cortisol (n = 67, 15%), fMRI (n = 29, 7%), skin conductivity (n = 26, 6%), fNIRs (n = 19, 4%), and pupillometry (n = 16, 4%). The majority of studies (n = 167, 38%) analyzed non-technical skills, followed by studies that analyzed technical skills (n = 155, 35%), knowledge (n = 114, 26%), and attitudinal skills (n = 61, 14%). 169 studies (38%) attempted to use physical or biophysical markers to differentiate between novice and expert. Conclusion This review provides a comprehensive description of the current use of physical and biophysical markers in medical education training, including the current technology and skills assessed. Additionally, while physical and biophysical markers have the potential to augment current assessment in medical education, there remains significant gaps in research surrounding reliability, validity, cost, practicality, and educational impact of implementing these markers of assessment.

Use the right words: evaluating the effect of word choice and word count on quality of narrative feedback in ophthalmology competency-based medical education assessments

Background: The purpose of this study was to investigate the effect of word choice on the quality of narrative feedback in ophthalmology resident trainee assessments following the introduction of competency-based medical education at Queen’s University. Methods: Assessment data from July 2017-December 2020 were retrieved from ElentraTM (Integrated Teaching and Learning Platform) and anonymized. Written feedback was assigned a Quality of Assessment for Learning (QuAL) score out of five based on this previously validated rubric. The correlation between QuAL score and specific coaching words was determined using a Spearman’s Rho analysis. Independent samples t-tests were used to compare the QuAL score when a specific word was used, and when it was absent. Results: A total of 1997 individual assessments were used in this analysis. The number of times the identified coaching words were used within a comment was significantly and positively associated with the total QuAL score, with the exception of “next time” (rho=0.039, p=0.082), “read” (rho = 0.036, p = 0.112), “read more” (rho = -0.025, p = 0.256) and “review” (rho = -0.017, p = 0.440). The strongest correlations were for “continue” (rho = 0.182, p &lt; 0.001), “try(ing)” (rho = 0.113, p &lt; 0.001) and “next step” (rho = 0.103, p &lt; 0.001). The mean value of the QuAL score increased when coaching words were used vs. not used with the largest mean difference of 1.44 (p &lt; 0.001) for “reflect”. A clear positive relationship was demonstrated between word count and QuAL score (rho = .556, p &lt; 0.001). Conclusions: The use of certain coaching words in written comments may improve the quality of feedback.

Role of technology enhanced assessment in medical and dental education: A scoping review

Background: Evolving innovative, feasible and cost effective technology related with teaching and learning in context of recent COVID-19 pandemic has led the medical educationists to think about its increased use in the assessment process. Need of the hour is to develop and establish a framework in our institutions for technology enhanced assessment (TEA) efficiently. European countries were using the e- assessment before the COVID-19 pandemic so their education system was less disturbed by it as compared to the Asian countries like Pakistan. TEA can be done with a number of tools and mobile phone technology, learning management systems (LMS) and social media have played remarkable role in this regard. The objective of this scoping review was to find best practice regarding the use of TEA in context of COVID-19 pandemic according to the Best Evidence Medical Education (BEME) principles. Methods: Databases which were searched were Google Scholar, Educational Resource Information System (ERIC), Cochrane library and ScienceDirect (Elsevier). The key words and phrases which we used for search were "TEA", "Online assessment", "assessment in context of COVID-19", "Role of TEA", "Advantages and disadvantages of TEA". 1252 articles were retrieved after screening total searched articles by Mendeley software. These articles were then assessed according to the eligibility criteria and 84 articles were selected out of which finally 12 articles were included in this scoping review after assessment of quality by Qualsyst appraisal tool. Data was extracted by using data extraction form which was synthesized to create the themes by axial coding. Results: After synthesis of data following themes were identified describing the TEA. What is TEA, role of TEA. Advantages and disadvantages of TEA, measurement and implementation issues in TEA. Prime benefit of TEA is its potential to foster learning by its ability to deliver vast variety of test formats in high fidelity virtual reality situations and by providing feedback after formative assessment using number of digital tools. Major issues related to TEA are cheating, validity, quality and security of test materials. Conclusion: E-assessment tools were already being used in European countries with the intent to improve the teaching and learning activities but after the COVID-19 pandemic it became the necessity. TEA has extreme potential for improving the learning and assessment process especially in context of current COVID-19 pandemic if the available technology is used efficiently. Implementation of TEA in place of conventional assessment should be tailored according to the local requirements and context. Finally the word of caution is to be prepared to address the issues of validity, quality, cheating and security of test materials connected with this technological change.

Demystifying AI: Current State and Future Role in Medical Education Assessment.

Medical education assessment faces multifaceted challenges, including data complexity, resource constraints, bias, feedback translation, and educational continuity. Traditional approaches often fail to adequately address these issues, creating stressful and inequitable learning environments. This article introduces the concept of precision education, a data-driven paradigm aimed at personalizing the educational experience for each learner. It explores how artificial intelligence (AI), including its subsets machine learning (ML) and deep learning (DL), can augment this model to tackle the inherent limitations of traditional assessment methods.AI can enable proactive data collection, offering consistent and objective assessments while reducing resource burdens. It has the potential to revolutionize not only competency assessment but also participatory interventions, such as personalized coaching and predictive analytics for at-risk trainees. The article also discusses key challenges and ethical considerations in integrating AI into medical education, such as algorithmic transparency, data privacy, and the potential for bias propagation.AI's capacity to process large datasets and identify patterns allows for a more nuanced, individualized approach to medical education. It offers promising avenues not only to improve the efficiency of educational assessments but also to make them more equitable. However, the ethical and technical challenges must be diligently addressed. The article concludes that embracing AI in medical education assessment is a strategic move toward creating a more personalized, effective, and fair educational landscape. This necessitates collaborative, multidisciplinary research and ethical vigilance to ensure that the technology serves educational goals while upholding social justice and ethical integrity.

Trainees' Perspectives on the Next Era of Assessment and Precision Education.

The next era of assessment in medical education promises new assessment systems, increased focus on ensuring high-quality equitable patient care, and precision education to drive learning and improvement. The potential benefits of using learning analytics and technology to augment medical training abound. To ensure that the ideals of this future for medical education are realized, educators should partner with trainees to build and implement new assessment systems. Coproduction of assessment systems by educators and trainees will help to ensure that new educational interventions are feasible and sustainable. In this paper, the authors provide a trainee perspective on 5 key areas that affect trainees in the next era of assessment: (1) precision education, (2) assessor education, (3) transparency in assessment development and implementation, (4) ongoing evaluation of the consequences of assessment, and (5) patient care data as sources of education outcomes.As precision education is developed, it is critical that trainees understand how their educational data are collected, stored, and ultimately utilized for educational outcomes. Since assessors play a key role in generating assessment data, it is important that they are prepared to give high-quality assessments and are continuously evaluated on their abilities. Transparency in the development and implementation of assessments requires communicating how assessments are created, the evidence behind them, and their intended uses. Furthermore, ongoing evaluation of the intended and unintended consequences that new assessments have on trainees should be conducted and communicated to trainees. Finally, trainees should participate in determining what patient care data are used to inform educational outcomes. The authors believe that trainee coproduction is critical to building stronger assessment systems that utilize evidence-based educational theories for improved learning and ultimately better patient care.

The Next Era of Assessment: Can Ensuring High-Quality, Equitable Patient Care Be the Defining Characteristic?

Previous eras of assessment in medical education have been defined by how assessment is done, from knowledge exams popularized in the 1960s to the emergence of work-based assessment in the 1990s to current efforts to integrate multiple types and sources of performance data through programmatic assessment. Each of these eras was a response to why assessment was performed (e.g., assessing medical knowledge with exams; assessing communication, professionalism, and systems competencies with work-based assessment). Despite the evolution of assessment eras, current evidence highlights the graduation of trainees with foundational gaps in the ability to provide high-quality care to patients presenting with common problems, and training program leaders report they graduate trainees they would not trust to care for themselves or their loved ones. In this article, the authors argue that the next era of assessment should be defined by why assessment is done: to ensure high-quality, equitable care. Assessment should place focus on demanding graduates possess the knowledge, skills, attitudes, and adaptive expertise to meet the needs of all patients and ensuring that graduates are able to do this in an equitable fashion. The authors explore 2 patient-focused assessment approaches that could help realize the promise of this envisioned era: entrustable professional activities (EPAs) and resident sensitive quality measures (RSQMs)/TRainee Attributable and Automatable Care Evaluations in Real-time (TRACERs). These examples illustrate how the envisioned next era of assessment can leverage existing and new data to provide precision education assessment that focuses on providing formative and summative feedback to trainees in a manner that seeks to ensure their learning outcomes prepare them to ensure high-quality, equitable patient outcomes.

Comparison of OBGYN postgraduate curricula and assessment methods between Canada and the Netherlands: an auto-ethnographic study.

Although the Dutch and the Canadian postgraduate Obstetrics and Gynecology (OBGYN) medical education systems are similar in their foundations [programmatic assessment, competency based, involving CanMED roles and EPAs (entrustable professional activities)] and comparable in healthcare outcome, their program structures and assessment methods considerably differ. We compared both countries' postgraduate educational blueprints and used an auto-ethnographic method to gain insight in the effects of training program structure and assessment methods on how trainees work. The research questions for this study are as follows: what are the differences in program structure and assessment program in Obstetrics and Gynecology postgraduate medical education in the Netherlands and Canada? And how does this impact the advancement to higher competency for the postgraduate trainee? We found four main differences. The first two differences are the duration of training and the number of EPAs defined in the curricula. However, the most significant difference is the way EPAs are entrusted. In Canada, supervision is given regardless of EPA competence, whereas in the Netherlands, being competent means being entrusted, resulting in meaningful and practical independence in the workplace. Another difference is that Canadian OBGYN trainees have to pass a summative written and oral exit examination. This difference in the assessment program is largely explained by cultural and legal aspects of postgraduate training, leading to differences in licensing practice. Despite the fact that programmatic assessment is the foundation for assessment in medical education in both Canada and the Netherlands, the significance of entrustment differs. Trainees struggle to differentiate between formative and summative assessments. The trainees experience both formative and summative forms of assessment as a judgement of their competence and progress. Based on this auto-ethnographic study, the potential for further harmonization of the OBGYN PGME in Canada and the Netherlands remains limited.

Does 'summative' count? The influence of the awarding of study credits on feedback use and test-taking motivation in medical progress testing.

Despite the increasing implementation of formative assessment in medical education, its' effect on learning behaviour remains questionable. This effect may depend on how students value formative, and summative assessments differently. Informed by Expectancy Value Theory, we compared test preparation, feedback use, and test-taking motivation of medical students who either took a purely formative progress test (formative PT-group) or a progress test that yielded study credits (summative PT-group). In a mixed-methods study design, we triangulated quantitative questionnaire data (n = 264), logging data of an online PT feedback system (n = 618), and qualitative interview data (n = 21) to compare feedback use, and test-taking motivation between the formative PT-group (n = 316), and the summative PT-group (n = 302). Self-reported, and actual feedback consultation was higher in the summative PT-group. Test preparation, and active feedback use were relatively low and similar in both groups. Both quantitative, and qualitative results showed that the motivation to prepare and consult feedback relates to how students value the assessment. In the interview data, a link could be made with goal orientation theory, as performance-oriented students perceived the formative PT as not important due to the lack of study credits. This led to low test-taking effort, and feedback consultation after the formative PT. In contrast, learning-oriented students valued the formative PT, and used it for self-study or self-assessment to gain feedback. Our results indicate that most students are less motivated to put effort in the test, and use feedback when there are no direct consequences. A supportive assessment environment that emphasizes recognition of the value of formative testing is required to motivate students to use feedback for learning.

ChatGPT to generate clinical vignettes for teaching and multiple-choice questions for assessment: A randomized controlled experiment

Aim This study aimed to evaluate the real-life performance of clinical vignettes and multiple-choice questions generated by using ChatGPT. Methods This was a randomized controlled study in an evidence-based medicine training program. We randomly assigned seventy-four medical students to two groups. The ChatGPT group received ill-defined cases generated by ChatGPT, while the control group received human-written cases. At the end of the training, they evaluated the cases by rating 10 statements using a Likert scale. They also answered 15 multiple-choice questions (MCQs) generated by ChatGPT. The case evaluations of the two groups were compared. Some psychometric characteristics (item difficulty and point-biserial correlations) of the test were also reported. Results None of the scores in 10 statements regarding the cases showed a significant difference between the ChatGPT group and the control group (p > .05). In the test, only six MCQs had acceptable levels (higher than 0.30) of point-biserial correlation, and five items could be considered acceptable in classroom settings. Conclusions The results showed that the quality of the vignettes are comparable to those created by human authors, and some multiple-questions have acceptable psychometric characteristics. ChatGPT has potential in generating clinical vignettes for teaching and MCQs for assessment in medical education.

Efficacy of Google Form–based MCQ tests for formative assessment in medical biochemistry education

BACKGROUND: Biochemistry, being a vast and complex subject, can be challenging for Phase I MBBS students to comprehend and retain. Embracing rapidly evolving technology can facilitate a more accessible learning experience. In this study, we investigated the effectiveness of using Google Form–based multiple-choice question (MCQ) tests as a formative assessment tool after each biochemistry lecture series. The aim was to assess the improvement and gather feedback of Phase I MBBS students on the utility of this assessment tool. MATERIALS AND METHODS: This educational prospective longitudinal study was conducted by the Department of Biochemistry at a university-affiliated medical college and tertiary care hospital. The study included 150 Phase I MBBS students as participants. Google Form–based MCQ tests were implemented as educational interventions after each lecture series during the study period. The study compared the internal assessment (IA) MCQ marks of students before and after the implementation of the intervention. In addition, feedback questionnaires were collected from the students. RESULTS: There was a significant improvement in students’ scores between the first IA (mean ± standard deviation [SD], 8.16 ± 3.08) and second IA (mean ± SD, 17.64 ± 2.02) (P &lt; 0.0001). According to students’ feedback, 149 out of 150 (99.3%) students found the use of Google Form–based MCQ tests as a formative assessment tool in the teaching–learning process to be highly beneficial and motivated them to engage in their biochemistry studies. CONCLUSION: With the shift toward competency-based medical education (CBME) in India, it is crucial for educators to embrace novel teaching–learning and evaluation approaches. Our study highlighted the efficacy of employing Google Form–based MCQ tests in enhancing students’ comprehension of the biochemistry subject, evaluating their scores and improving the overall quality of learning. Through this mode of assessment, teachers were able to provide targeted feedback on areas that required improvement, thereby enhancing the learning experience.

Fostering Empathy: Nurturing Compassionate Healers through Innovative Teaching and Insightful Assessment in Medical Education

Abstract: The significance of empathy in the realm of medical education cannot be undermined, as once a student embarks on their journey to serve and heal mankind, cultivating empathetic skills becomes an indispensable part of their training. Empathy plays a significant role in healthcare delivery in the domains of patient outcomes and doctor-patient relationships. This article provides a brief review of empathy education and assessment techniques in the field of medicine. Upon realizing the merits that can be attributed to empathy, it is of paramount importance that medical students must be trained in empathy to transform them into patient-centered healthcare professionals. Only training medical students in empathy will not serve the purpose, and we must complement the training component with assessment. In conclusion, empathy is one of the key components of compassionate and patient-centered medical care. This calls for the need to train and assess medical students during their training period to improve their empathetic abilities and support the process of producing competent and compassionate future medical doctors.

Technical Skills Curriculum in Neonatology: A Modified European Delphi Study

Introduction: Simulation-based training (SBT) aids healthcare providers in acquiring the technical skills necessary to improve patient outcomes and safety. However, since SBT may require significant resources, training all skills to a comparable extent is impractical. Hence, a strategic prioritization of technical skills is necessary. While the European Training Requirements in Neonatology provide guidance on necessary skills, they lack prioritization. We aimed to identify and prioritize technical skills for a SBT curriculum in neonatology. Methods: A three-round modified Delphi process of expert neonatologists and neonatal trainees was performed. In round one, the participants listed all the technical skills newly trained neonatologists should master. The content analysis excluded duplicates and non-technical skills. In round two, the Copenhagen Academy for Medical Education and Simulation Needs Assessment Formula (CAMES-NAF) was used to preliminarily prioritize the technical skills according to frequency, importance of competency, SBT impact on patient safety, and feasibility for SBT. In round three, the participants further refined and reprioritized the technical skills. Items achieving consensus (agreement of ≥75%) were included. Results: We included 168 participants from 10 European countries. The response rates in rounds two and three were 80% (135/168) and 87% (117/135), respectively. In round one, the participants suggested 1964 different items. Content analysis revealed 81 unique technical skills prioritized in round two. In round three, 39 technical skills achieved consensus and were included. Conclusion: We reached a European consensus on a prioritized list of 39 technical skills to be included in a SBT curriculum in neonatology.

The Impact of the COVID-19 Pandemic on Clinical Education: A Descriptive Phenomenological Study

Background: The COVID-19 pandemic has affected educational processes and methods. Identifying and sharing educational experiences helps policymaking and educational decision-making and planning to face similar situations at the national and international levels. Objectives: This study aimed to explore medical students' lived experiences about the effect of the COVID-19 pandemic on their clinical education. Methods: The main question of the research was, “what were the experiences of medical students from training in the clinical environment during the days of COVID-19?” Therefore, this descriptive phenomenological study was conducted at Mazandaran University of Medical Sciences, Mazandaran, Iran, in 2021. The semi-structured virtual interviews were conducted with 18 medical students. Data were analyzed using Colaizz's method. Results: Most participants were female (N = 10), and their average age was 23.71 ± 2.06. The essential experiences were related to three themes and 20 sub-themes. Themes highlighted teaching and learning methods, educational planning and management disruptions, and assessment methods. Virtual teaching and learning included virtual journal clubs, conferences, tumor boards, case reports, morning reports, skill labs, small groups, flipped classroom methods, etc. Disruption in educational planning was mainly observed in rounds and significant rotations. OSCE, logbook, and activity in e-learning were the predominant assessments. Conclusions: Based on the results, three themes were based on medical students’ experiences. According to these themes, COVID-19 was an accelerator for shifting the paradigm in Iran's medical education and students' learning and assessment strategies. Therefore, some departments used COVID-19 as an opportunity and adapted to this shift. However, several departments were disturbed in their planning and management. requiring support and training to face similar situations.

Using Natural Language Processing to Evaluate the Quality of Supervisor Narrative Comments in Competency-Based Medical Education.

Learner development and promotion rely heavily on narrative assessment comments, but narrative assessment quality is rarely evaluated in medical education. Educators have developed tools such as the Quality of Assessment for Learning (QuAL) tool to evaluate the quality of narrative assessment comments; however, scoring the comments generated in medical education assessment programs is time intensive. The authors developed a natural language processing (NLP) model for applying the QuAL score to narrative supervisor comments. A sample of 2,500 Entrustable Professional Activities assessments were randomly extracted and deidentified from the McMaster (1,250 comments) and Saskatchewan (1,250 comments) emergency medicine (EM) residency training programs during the 2019-2020 academic year. Comments were rated using the QuAL score by 25 EM faculty members and 25 EM residents. The results were used to develop and test an NLP model to predict the overall QuAL score and QuAL subscores. All 50 raters completed the rating exercise. Approximately 50% of the comments had perfect agreement on the QuAL score, with the remaining resolved by the study authors. Creating a meaningful suggestion for improvement was the key differentiator between high- and moderate-quality feedback. The overall QuAL model predicted the exact human-rated score or 1 point above or below it in 87% of instances. Overall model performance was excellent, especially regarding the subtasks on suggestions for improvement and the link between resident performance and improvement suggestions, which achieved 85% and 82% balanced accuracies, respectively. This model could save considerable time for programs that want to rate the quality of supervisor comments, with the potential to automatically score a large volume of comments. This model could be used to provide faculty with real-time feedback or as a tool to quantify and track the quality of assessment comments at faculty, rotation, program, or institution levels.

Deidentifying Narrative Assessments to Facilitate Data Sharing in Medical Education.

Narrative assessments are commonly incorporated into competency-based medical education programs. However, efforts to share competency-based medical education assessment data among programs to support the evaluation and improvement of assessment systems have been limited in part because of security concerns. Deidentifying assessment data mitigates these concerns, but deidentifying narrative assessments is time-consuming, resource intensive, and error prone. The authors developed and tested a tool to automate the deidentification of narrative assessments and facilitate their review. The authors met throughout 2021 and 2022 to iteratively design, test, and refine the deidentification algorithm and data review interface. Preliminary testing of the prototype deidentification algorithm was performed using narrative assessments from the University of Saskatchewan emergency medicine program. The algorithm's accuracy was assessed by the authors using the review interface designed for this purpose. Formal testing included 2 rounds of deidentification and review by members of the authorship team. Both the algorithm and data review interface were refined during the testing process. Authors from 3 institutions, including 3 emergency medicine programs, an anesthesia program, and a surgical program, participated in formal testing. In the final round of review, 99.4% of the narrative assessments were fully deidentified (names, nicknames, and pronouns removed). The results were comparable for each institution and specialty. The data review interface was improved with feedback obtained after each round of review and found to be intuitive.Next StepsThis innovation has demonstrated viability evidence of an algorithmic approach to the deidentification of assessment narratives while reinforcing that a small number of errors are likely to persist. Future steps include the refinement of both the algorithm to improve its accuracy and the data review interface to support additional data set formats.

COVID-19 impact on the assessment methodology of undergraduate medical students: a systematic review of the lessons learned

The COVID-19 pandemic had a substantial social, economic, political, and educational impact worldwide. Due to the social contact restrictions, areas such as medical education were highly affected. Assessment in medical education, was already a sensitive topic, and it proved to be even more challenging as different teaching and learning contexts required huge adaptations in a short period. This systematic review provides an overview of the impact of COVID-19 on the assessment of medical students and can serve as a reference to improve this area. We conducted the review based on the PRISMA tool and searched in PubMed, EBSCO, and ScienceDirect. Studies describing the assessment methodologies used during the pandemic were included. Of the 501 initial articles, 18 were included in this review. Collected data was based on the regime, subject, teaching/assessment methodologies, platforms used, grades, students’ and teachers’ perceptions, and measures to prevent academic dishonesty. The results suggest that technology played a central role during the pandemic, and universities were concerned about the transition to online learning regarding teaching and assessment, but students and teachers should be prepared for it. Formative assessment methodologies and feedback were emphasized, and summative tools were adapted to prevent fraud. Students and teachers were generally satisfied with online learning and assessment, which had no significant difference in the examination scores, but they preferred conventional teaching. The COVID-19 pandemic brought an opportunity to analyze and rethink the medical curriculum. Thus, further investigations are needed on combining traditional and online teaching strategies and emphasis on the assessment.

Programmatic assessment in Medical Education – Review

A programmatic assessment is a systematic process by which we evaluate our processes, services, and outcomes through a systematic approach with the intention of enhancing our quality and performance. With the help of a structured approach, we will be able to gather, analyze, and interpret data in order to determine strengths and weaknesses within our company's operations. As part of the organization's assessment program, a variety of methodologies will be used, including surveys, performance metrics, and peer reviews, to provide an understanding of the effectiveness of the organization as a whole. In this review, the primary objectives are to identify the benefits, the approach to their implementation, as well as areas for improvement, set measurable goals, develop strategic changes, and identify the challenges associated with implementing programmatic assessments in medical education. Through the adoption of a continuous improvement culture, this assessment program will have the ability to empower our organization to adapt to evolving challenges as well as provide better services to our stakeholders.

Exploring gender and thematic differences in qualitative assessments of internal medicine resident performance

IntroductionEvidence suggests gender disparities in medical education assessment, including differences in ratings of competency and narrative comments provided in resident performance assessments. This study explores how gender manifests within the content of qualitative assessments (i.e., narrative comments or performance feedback) of resident performance.MethodsQualitative content analysis was used to explore gender-based differences in narrative comments included in faculty assessments of resident performance during inpatient medicine rotations at six Internal Medicine residency programs, 2016–2017. A blinded, multi-analyst approach was employed to identify themes across comments. Patterns in themes with resident gender and post-graduate year (PGY) were explored, focusing on PGY2 and PGY3 when residents are serving in the team leader role.ResultsData included 3,383 evaluations with narrative comments of 385 men (55.2%) and 313 women residents (44.8%). There were thematic differences in narrative comments received by men and women residents and how these themes manifested within comments changed with training time. Compared to men, comments about women had a persistent relationship-orientation and emphasized confidence over training including as interns and in PGY2 and PGY3, when serving as team leader. The relationship-orientation was characterized not only by the residents’ communal attributes but also their interpersonal and communication skills, including efforts supporting others and establishing the tone for the team. Comments about women residents often highlighted confidence, including recommendations around behaviors that convey confidence in decision-making and team leadership.DiscussionThere were gender-based thematic differences in qualitative assessments. Comments about women resident team leaders highlight relationship building skills and urge confidence and actions that convey confidence as team leader. Persistent attention to communal skills suggests gendered expectations for women resident team leaders and a lost opportunity for well-rounded feedback to the disadvantage of women residents. These findings may inform interventions to promote equitable assessment, such as providing feedback across the competencies.

The Heart of the Matter: Immune Checkpoint Inhibitors and Immune-Related Adverse Events on the Cardiovascular System.

Cancer remains a prominent global cause of mortality, second only to cardiovascular disease. The past decades have witnessed substantial advancements in anti-cancer therapies, resulting in improved outcomes. Among these advancements, immunotherapy has emerged as a promising breakthrough, leveraging the immune system to target and eliminate cancer cells. Despite the remarkable potential of immunotherapy, concerns have arisen regarding associations with adverse cardiovascular events. This review examines the complex interplay between immunotherapy and cardiovascular toxicity and provides an overview of immunotherapy mechanisms, clinical perspectives, and potential biomarkers for adverse events, while delving into the intricate immune responses and evasion mechanisms displayed by cancer cells. The focus extends to the role of immune checkpoint inhibitors in cancer therapy, including CTLA-4, PD-1, and PD-L1 targeting antibodies. This review underscores the multifaceted challenges of managing immunotherapy-related cardiovascular toxicity. Risk factors for immune-related adverse events and major adverse cardiac events are explored, encompassing pharmacological, treatment-related, autoimmune, cardiovascular, tumor-related, social, genetic, and immune-related factors. The review also advocates for enhanced medical education and risk assessment tools to identify high-risk patients for preventive measures. Baseline cardiovascular evaluations, potential prophylactic strategies, and monitoring of emerging toxicity symptoms are discussed, along with the potential of adjunct anti-inflammatory therapies.